Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization

In recent years, decentralized learning has emerged as a powerful tool not only for large-scale machine learning, but also for preserving privacy. One of the key challenges in decentralized learning is that the data distribution held by each node is statistically heterogeneous. To address this challenge, the primal-dual algorithm called the Edge-Consensus Learning (ECL) was proposed and was experimentally shown to be robust to the heterogeneity of data distributions. However, the convergence rate of the ECL is provided only when the objective function is convex, and has not been shown in a standard machine learning setting where the objective function is non-convex. Furthermore, the intuitive reason why the ECL is robust to the heterogeneity of data distributions has not been investigated. In this work, we first investigate the relationship between the ECL and Gossip algorithm and show that the update formulas of the ECL can be regarded as correcting the local stochastic gradient in the Gossip algorithm. Then, we propose the Generalized ECL (G-ECL), which contains the ECL as a special case, and provide the convergence rates of the G-ECL in both (strongly) convex and non-convex settings, which do not depend on the heterogeneity of data distributions. Through synthetic experiments, we demonstrate that the numerical results of both the G-ECL and ECL coincide with the convergence rate of the G-ECL

Embarrassingly Simple Text Watermarks

We propose Easymark, a family of embarrassingly simple yet effective watermarks. Text watermarking is becoming increasingly important with the advent of Large Language Models (LLM). LLMs can generate texts that cannot be distinguished from human-written texts. This is a serious problem for the credibility of the text. Easymark is a simple yet effective solution to this problem. Easymark can inject a watermark without changing the meaning of the text at all while a validator can detect if a text was generated from a system that adopted Easymark or not with high credibility. Easymark is extremely easy to implement so that it only requires a few lines of code. Easymark does not require access to LLMs, so it can be implemented on the user-side when the LLM providers do not offer watermarked LLMs. In spite of its simplicity, it achieves higher detection accuracy and BLEU scores than the state-of-the-art text watermarking methods. We also prove the impossibility theorem of perfect watermarking, which is valuable in its own right. This theorem shows that no matter how sophisticated a watermark is, a malicious user could remove it from the text, which motivate us to use a simple watermark such as Easymark. We carry out experiments with LLM-generated texts and confirm that Easymark can be detected reliably without any degradation of BLEU and perplexity, and outperform state-of-the-art watermarks in terms of both quality and reliability

Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data

SGD with momentum acceleration is one of the key components for improving the performance of neural networks. For decentralized learning, a straightforward approach using momentum acceleration is Distributed SGD (DSGD) with momentum acceleration (DSGDm). However, DSGDm performs worse than DSGD when the data distributions are statistically heterogeneous. Recently, several studies have addressed this issue and proposed methods with momentum acceleration that are more robust to data heterogeneity than DSGDm, although their convergence rates remain dependent on data heterogeneity and decrease when the data distributions are heterogeneous. In this study, we propose Momentum Tracking, which is a method with momentum acceleration whose convergence rate is proven to be independent of data heterogeneity. More specifically, we analyze the convergence rate of Momentum Tracking in the standard deep learning setting, where the objective function is non-convex and the stochastic gradient is used. Then, we identify that it is independent of data heterogeneity for any momentum coefficient $\beta\in [0, 1)$. Through image classification tasks, we demonstrate that Momentum Tracking is more robust to data heterogeneity than the existing decentralized learning methods with momentum acceleration and can consistently outperform these existing methods when the data distributions are heterogeneous

EGUIDE project and treatment guidelines

Aim: Although treatment guidelines for pharmacological therapy for schizophrenia and major depressive disorder have been issued by the Japanese Societies of Neuropsychopharmacology and Mood Disorders, these guidelines have not been well applied by psychiatrists throughout the nation. To address this issue, we developed the ‘Effectiveness of Guidelines for Dissemination and Education in Psychiatric Treatment (EGUIDE)’ integrated education programs for psychiatrists to disseminate the clinical guidelines. Additionally, we conducted a systematic efficacy evaluation of the programs. Methods: Four hundred thirteen out of 461 psychiatrists attended two 1‐day educational programs based on the treatment guidelines for schizophrenia and major depressive disorder from October 2016 to March 2018. We measured the participants’ clinical knowledge of the treatment guidelines using self‐completed questionnaires administered before and after the program to assess the effectiveness of the programs for improving knowledge. We also examined the relation between the participants’ demographics and their clinical knowledge scores. Results: The clinical knowledge scores for both guidelines were significantly improved after the program. There was no correlation between clinical knowledge and participant demographics for the program on schizophrenia; however, a weak positive correlation was found between clinical knowledge and the years of professional experience for the program on major depressive disorder. Conclusion: Our results provide evidence that educational programs on the clinical practices recommended in guidelines for schizophrenia and major depressive disorder might effectively improve participants’ clinical knowledge of the guidelines. These data are encouraging to facilitate the standardization of clinical practices for psychiatric disorders

Insights into Land Plant Evolution Garnered from the Marchantia polymorpha Genome.

The evolution of land flora transformed the terrestrial environment. Land plants evolved from an ancestral charophycean alga from which they inherited developmental, biochemical, and cell biological attributes. Additional biochemical and physiological adaptations to land, and a life cycle with an alternation between multicellular haploid and diploid generations that facilitated efficient dispersal of desiccation tolerant spores, evolved in the ancestral land plant. We analyzed the genome of the liverwort Marchantia polymorpha, a member of a basal land plant lineage. Relative to charophycean algae, land plant genomes are characterized by genes encoding novel biochemical pathways, new phytohormone signaling pathways (notably auxin), expanded repertoires of signaling pathways, and increased diversity in some transcription factor families. Compared with other sequenced land plants, M. polymorpha exhibits low genetic redundancy in most regulatory pathways, with this portion of its genome resembling that predicted for the ancestral land plant. PAPERCLIP

Observation of Magnetic Domains in Amorphous Magnetic Wires with a Diameter of 10 μm Used in GSR Sensors

The core of a Gigahertz Spin Rotation (GSR) sensor, a compact and highly sensitive magnetic sensor, is composed of Co–Fe-based amorphous magnetic wire with a diameter of 10 μm. Observations of the magnetic domain structure showed that this magnetic wire has unusual magnetic noise characteristics. Bamboo-shaped magnetic domains a few hundred micrometers in width were observed to form inside the wire, and smaller domains a few micrometers across were observed to form inside these larger domains. The magnetic domain pattern changed abruptly when an external magnetic field was applied to the wire. Herein is shown how these changes may be a source of magnetic noise in the wire

The Pattern of Sexual Interest of Female-to-Male Transsexual Persons With Gender Identity Disorder Does Not Resemble That of Biological Men: An Eye-Tracking Study

Introduction: Very little has been elucidated about sexual interest in female-to-male (FtM) transsexual persons. Aims: To investigate the sexual interest of FtM transsexual persons vs that of men using an eye-tracking system. Methods: The study included 15 men and 13 FtM transsexual subjects who viewed three sexual videos (clip 1: sexy clothed young woman kissing the region of the male genitals covered by underwear; clip 2: naked actor and actress kissing and touching each other; and clip 3: heterosexual intercourse between a naked actor and actress) in which several regions were designated for eye-gaze analysis in each frame. The designation of each region was not visible to the participants. Main Outcome Measures: Visual attention was measured across each designated region according to gaze duration. Results: For clip 1, there was a statistically significant sex difference in the viewing pattern between men and FtM transsexual subjects. Longest gaze time was for the eyes of the actress in men, whereas it was for non-human regions in FtM transsexual subjects. For clip 2, there also was a statistically significant sex difference. Longest gaze time was for the face of the actress in men, whereas it was for non-human regions in FtM transsexual subjects, and there was a significant difference between regions with longest gaze time. The most apparent difference was in the gaze time for the body of the actor: the percentage of time spent gazing at the body of the actor was 8.35% in FtM transsexual subjects, whereas it was only 0.03% in men. For clip 3, there were no statistically significant differences in viewing patterns between men and FtM transsexual subjects, although longest gaze time was for the face of the actress in men, whereas it was for non-human regions in FtM transsexual subjects. Conclusion: We suggest that the characteristics of sexual interest of FtM transsexual persons are not the same as those of biological men. Tsujimura A, Kiuchi H, Soda T, et al. The Pattern of Sexual Interest of Female-to-Male Transsexual Persons With Gender Identity Disorder Does Not Resemble That of Biological Men: An Eye-Tracking Study. Sex Med 2017;5:e169–e174